A Monte-Carlo Policy Rollout Planner for Pathfinding in Real-Time Strategy (RTS) Games

In this paper, we present a novel Monte-Carlo policy rollout algorithm (called MCRT-GAP) that uses a hybrid of focussed exploration of new actions and exploitation of promising actions to expand the lookahead search from the current state in a Monte-Carlo simulation model. The initial estimates of the action values are computed using a combination of the distance heuristic and a collision estimate. To balance the exploration of new actions and the exploitation of already explored promising actions, MCRT-GAP uses a greedy action selection approach to exploit the best action and a random selection approach to explore new actions within a small set of useful actions (i.e. smaller than the size of the action set in a domain world). This small set is called a corridor. In this paper, we describe the motivation and the algorithmic details of MCRTGAP. The paper also presents the theoretical properties